feat: allow configuring relay-proxy request log verbosity #1946

fals · 2024-05-31T14:38:50Z

Description

TL;DR;
We are using the relay-proxy for a while in our system and we noticed that every request to relay proxy is recorded as log with level=info. Currently this log became a problem, because we generate 1.8 million request every 5 min to relay-proxy. The amount of logs injected in increase as consequence storage and cost.

This PR implements configuring the verbosity of relay-proxy, allowing you to log more if needed, or only errors.

Checklist

I have tested this code
I have added unit test to cover this code
I have updated the documentation (README.md and /website/docs)
I have followed the contributing guide

netlify · 2024-05-31T14:39:07Z

✅ Deploy Preview for go-feature-flag-doc-preview ready!

Name	Link
🔨 Latest commit	`0054edf`
🔍 Latest deploy log	https://app.netlify.com/sites/go-feature-flag-doc-preview/deploys/6668aaede75ecf0008e3e90f
😎 Deploy Preview	https://deploy-preview-1946--go-feature-flag-doc-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

fals

allow select verbosity of relay proxy request logging

thomaspoignant

Hello @fals,
thanks a lot for this PR and I think you are right but we should probably be better in log management in general.

Instead of the VerboseRequestLogging I would prefer to have a property called LogLevel to set the level of the logs we want for the application.
The levels should be the classic ones debug, info, warn, error.

By setting this new option, we can also maybe deprecate the debug that is a bit too trivial here.

cmd/relayproxy/api/middleware/zap.go

codecov · 2024-06-03T09:28:23Z

Codecov Report

Attention: Patch coverage is 74.46809% with 12 lines in your changes missing coverage. Please review.

Project coverage is 86.88%. Comparing base (2c71961) to head (0054edf).

Files	Patch %	Lines
cmd/relayproxy/main.go	0.00%	9 Missing ⚠️
cmd/editor/main.go	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1946      +/-   ##
==========================================
+ Coverage   86.81%   86.88%   +0.07%     
==========================================
  Files          98       98              
  Lines        3526     3553      +27     
==========================================
+ Hits         3061     3087      +26     
- Misses        357      358       +1     
  Partials      108      108

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

thomaspoignant

Sorry for the time it took for me to review it.
I added some comments about:

the removal of the debug option.
the way you init zap.
the name of the option that is not using camelcase in the config file.

Despite that it looks super good 👍

cmd/relayproxy/config/config.go

cmd/relayproxy/testdata/config/valid-file.yaml

cmd/relayproxy/testdata/config/validate-array-env-file.yaml

cmd/relayproxy/testdata/config/valid-file.json

cmd/relayproxy/api/middleware/zap.go

cmd/relayproxy/config/config.go

cmd/relayproxy/testdata/config/valid-file.toml

cmd/relayproxy/log/logger.go

Co-authored-by: Thomas Poignant <[email protected]>

fals · 2024-06-10T10:07:27Z

Sorry for the time it took for me to review it. I added some comments about:

the removal of the debug option.

the way you init zap.

the name of the option that is not using camelcase in the config file.

Despite that it looks super good 👍

fals · 2024-06-10T10:30:57Z

cmd/relayproxy/main.go

@@ -48,37 +48,40 @@ func main() {
 	_ = f.Parse(os.Args[1:])

 	// Init logger
-	zapLog := log.InitLogger()
-	defer func() { _ = zapLog.Sync() }()
+	logger := log.InitLogger()


@thomaspoignant any tip to satisfy this coverage? The test for main is empty, just a dummy test there, and all new code I introduced is flagged, so I believe I have to find where exactly you test this cases. Could you guide me?

Most of the time I don't test the main function.
So I guess this is not really a problem here.

@thomaspoignant let me know then what else needs to change to push this forward. The codecoverage isn't passing, that's why I asked the question.

thomaspoignant

I have spotted 2 minor things and I have pushed 1 more commit.
I will merge the PR.

Thanks a lot @fals for your contribution, this will be available in the next version of GO Feature Flag.
To continue improving the logging of GOFF I have also opened this PR #1955 that will support levelled log inside the core go module of GOFF.

thomaspoignant · 2024-06-11T19:16:54Z

cmd/relayproxy/config/config.go

+	if proxyConf.Debug && proxyConf.LogLevel == "" {
+		log.Warn(
+			"Option Debug that you are using in your configuration file is deprecated" +
+				"and will be removed in future versions." +
+				"Please use logLevel: debug to continue to run the relay-proxy with debug logs.")
+	}


proxyConf.LogLevel == "" will never be true because you have a default value.

Suggested change

if proxyConf.Debug && proxyConf.LogLevel == "" {

log.Warn(

"Option Debug that you are using in your configuration file is deprecated" +

"and will be removed in future versions." +

"Please use logLevel: debug to continue to run the relay-proxy with debug logs.")

}

if proxyConf.Debug {

log.Warn(

"Option Debug that you are using in your configuration file is deprecated " +

"and will be removed in future versions." +

"Please use logLevel: debug to continue to run the relay-proxy with debug logs.")

}

thomaspoignant · 2024-06-11T19:43:21Z

cmd/relayproxy/config/config.go

+	if c == nil {
+		return false
+	}
+	return c.LogLevel == "debug" || c.Debug


Checking in lowercase to be ready for any configuration string.

Suggested change

return c.LogLevel == "debug" || c.Debug

return strings.ToLower(c.LogLevel) == "debug" || c.Debug

Signed-off-by: Thomas Poignant <[email protected]>

sonarcloud · 2024-06-11T19:52:51Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

feature: allow configuring relay-proxy request log verbosity

50fba98

github-actions bot assigned thomaspoignant May 31, 2024

github-actions bot requested a review from thomaspoignant May 31, 2024 14:39

fals commented May 31, 2024

View reviewed changes

fals changed the title ~~feature: allow configuring relay-proxy request log verbosity~~ feat: allow configuring relay-proxy request log verbosity Jun 3, 2024

thomaspoignant reviewed Jun 3, 2024

View reviewed changes

cmd/relayproxy/api/middleware/zap.go Outdated Show resolved Hide resolved

fals added 2 commits June 3, 2024 11:25

fix test setup

79f1a97

linting

075f6e1

fals added 11 commits June 3, 2024 11:57

Merge branch 'main' into relay-proxy-verbosity

5d86d0a

tweek zap level

8388af3

remove variable from test

99d1cbf

cover log level

5ae6459

log level coverage

2befa25

just forward existing error from zap

4f19c87

code coverage

ebc8dff

set log level

4c7f7b2

broke other service

21c8ae8

force build

0ba21fa

use new logger

1dc712e

thomaspoignant requested changes Jun 7, 2024

View reviewed changes

fals and others added 6 commits June 10, 2024 10:13

Apply suggestions from code review

622ccb3

Co-authored-by: Thomas Poignant <[email protected]>

pr remarks

db3ec04

should be backwards compatible

4c03016

wrong log level for testing

c8e94f2

lll linter

cf5880b

coverage concerns

aa88ef7

fals closed this Jun 10, 2024

fals reopened this Jun 10, 2024

github-actions bot requested a review from thomaspoignant June 10, 2024 10:07

use logger instead

87afd61

fals commented Jun 10, 2024

View reviewed changes

Merge branch 'main' into relay-proxy-verbosity

e4e9ba7

thomaspoignant approved these changes Jun 11, 2024

View reviewed changes

thomaspoignant added 2 commits June 11, 2024 21:51

fix: Support all cases for debug + fix warn message

28936c3

Signed-off-by: Thomas Poignant <[email protected]>

Merge branch 'main' into relay-proxy-verbosity

0054edf

thomaspoignant merged commit a2e12be into thomaspoignant:main Jun 11, 2024
21 of 22 checks passed

BrewTestBot mentioned this pull request Jun 12, 2024

go-feature-flag-relay-proxy 1.30.0 Homebrew/homebrew-core#174383

Merged

thomaspoignant pushed a commit that referenced this pull request Sep 5, 2024

feat: allow configuring relay-proxy request log verbosity (#1946)

d105b19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: allow configuring relay-proxy request log verbosity #1946

feat: allow configuring relay-proxy request log verbosity #1946

fals commented May 31, 2024 •

edited

Loading

netlify bot commented May 31, 2024 •

edited

Loading

fals left a comment

thomaspoignant left a comment

codecov bot commented Jun 3, 2024 •

edited

Loading

thomaspoignant left a comment

fals commented Jun 10, 2024

fals Jun 10, 2024

thomaspoignant Jun 10, 2024

fals Jun 11, 2024 •

edited

Loading

thomaspoignant left a comment

thomaspoignant Jun 11, 2024

thomaspoignant Jun 11, 2024

sonarcloud bot commented Jun 11, 2024

	return c.LogLevel == "debug" \|\| c.Debug
	return strings.ToLower(c.LogLevel) == "debug" \|\| c.Debug

feat: allow configuring relay-proxy request log verbosity #1946

feat: allow configuring relay-proxy request log verbosity #1946

Conversation

fals commented May 31, 2024 • edited Loading

Description

Checklist

netlify bot commented May 31, 2024 • edited Loading

✅ Deploy Preview for go-feature-flag-doc-preview ready!

fals left a comment

Choose a reason for hiding this comment

thomaspoignant left a comment

Choose a reason for hiding this comment

codecov bot commented Jun 3, 2024 • edited Loading

Codecov Report

thomaspoignant left a comment

Choose a reason for hiding this comment

fals commented Jun 10, 2024

fals Jun 10, 2024

Choose a reason for hiding this comment

thomaspoignant Jun 10, 2024

Choose a reason for hiding this comment

fals Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

thomaspoignant left a comment

Choose a reason for hiding this comment

thomaspoignant Jun 11, 2024

Choose a reason for hiding this comment

thomaspoignant Jun 11, 2024

Choose a reason for hiding this comment

sonarcloud bot commented Jun 11, 2024

Quality Gate passed

fals commented May 31, 2024 •

edited

Loading

netlify bot commented May 31, 2024 •

edited

Loading

codecov bot commented Jun 3, 2024 •

edited

Loading

fals Jun 11, 2024 •

edited

Loading